Interactive environment with virtual environment space scanning

ABSTRACT

An interactive environment image may be displayed in a virtual environment space, and interaction with the interactive environment image may be detected within a three-dimensional space that corresponds to the virtual environment space. The interactive environment image may be a three-dimensional image, or it may be two-dimensional. An image is displayed to provide a visual representation of an interactive environment image including one or more virtual objects, which may be spatially positioned. User interaction with the visualized representation in the virtual environment space may be detected and, in response to user interaction, the interactive environment image may be changed.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/883,972, filed on May 26, 2020 and titled INTERACTIVE ENVIRONMENTWITH THREE-DIMENSIONAL SCANNING, (“the '972 application”), now U.S. Pat.No. 10,928,958, issued Feb. 23, 2021, which is a continuation of U.S.patent application Ser. No. 16/519,593, filed on Jul. 23, 2019 andtitled PROJECTED, INTERACTIVE ENVIRONMENT, (“the '593 application”), nowU.S. Pat. No. 10,664,105, issued May 26, 2020, which is a continuationof U.S. patent application Ser. No. 15/980,638, filed on May 15, 2018and titled PROJECTED, INTERACTIVE ENVIRONMENT (“the '638 application”),now U.S. Pat. No. 10,359,888, issued Jul. 23, 2019. The '638 applicationis a continuation of U.S. patent application Ser. No. 15/414,617, filedon Jan. 24, 2017 and titled PROJECTION OF INTERACTIVE ENVIRONMENT (“the'617 application”), now U.S. Pat. No. 9,971,458, issued May 15, 2018.The '617 application is a continuation-in-part of U.S. patentapplication Ser. No. 14/462,750, filed on Aug. 19, 2014 and titledPROJECTION OF INTERACTIVE ENVIRONMENT (“the '750 application”), now U.S.Pat. No. 9,550,124, issued Jan. 24, 2017. The '750 application is acontinuation of U.S. patent application Ser. No. 13/547,626, filed onJul. 12, 2012 and titled PROJECTION OF INTERACTIVE GAME ENVIRONMENT(“the '626 application”), now U.S. Pat. No. 8,808,089, issued Aug. 19,2014. The '626 application is a continuation-in-part of U.S. patentapplication Ser. No. 12/855,604, filed on Aug. 12, 2010 and titledPROJECTION OF INTERACTIVE GAME ENVIRONMENT (“the '604 application”),abandoned. The '604 application is a continuation-in-part of U.S. patentapplication Ser. No. 12/651,947, filed on Jan. 4, 2010 and titledELECTRONIC CIRCLE GAME SYSTEM (“the '947 application”), abandoned. The'947 application is a continuation-in-part of U.S. patent applicationSer. No. 12/411,289, filed on Mar. 25, 2009 and titled WIRELESSLYDISTRIBUTED ELECTRONIC CIRCLE GAMING (“the '289 application”),abandoned.

The entire disclosures of the '972 application, the '593 application,the '638 application, the '617 application, the '750 application, the'626 application, the '604 application, the '947 application and the'289 application are, by this reference, incorporated herein.

SUMMARY

Embodiments described herein relate to the projection of an image and tointeraction with the projected image. The image may be a two-dimensionalimage, or it may be three-dimensional. Data is received that representsvirtual objects that are spatially positioned in virtual environmentspace. An image is then projected to display a visual representation ofall or a portion of the virtual environment space, including one or moreof the virtual objects within the virtual environment space. The systemmay then detect user interaction with the locations of one or more ofthe virtual objects in the virtual environment space, as seen in thevisual representation of the virtual environment space displayed by theimage that has been projected and, in response thereto, change the imagethat is projected. That interaction may be via an input device, or evenmore directly via interaction with the projected image. In the case ofdirect interaction, the user might interact with a virtual object withinthe virtual environment space, or with a physical object (e.g., a gamepiece, a game board, etc.) that is within the virtual environment spacevisually represented by the image. Thus, a user may interact withvisualized representations of virtual environment space, enablingcomplex and interesting interactivity scenarios and applications.

Systems that project images that represent virtual environment spacesand virtual objects themselves, and that detect interaction with one ormore of the virtual objects are also disclosed. Such a system, which mayalso be referred to herein as a “projection system,” may be capable ofmodifying an image based on that interaction, and may be capable ofprojecting the modified image to display the result(s) of suchinteraction.

This summary is not intended to identify key features or essentialfeatures of the claimed subject matter, nor is it intended to be used asan aid in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above-recited and otheradvantages and features can be obtained, a more particular descriptionof various embodiments will be provided by reference to the accompanyingdrawings. Understanding that the drawings depict only sample embodimentsand are not, therefore, to be considered to be limiting of the scope ofany of the appended claims, the embodiments will be described andexplained with additional specificity and detail through the use of theaccompanying drawings in which:

FIG. 1 abstractly illustrates an embodiment of a distributed system thatincludes an embodiment of an interactive projection system;

FIG. 2 abstractly illustrates an interactive image projection systemthat represents an embodiment of the interactive image projection systemof FIG. 1;

FIG. 3 illustrates an example embodiment of a virtual environment spacethat includes virtual objects;

FIG. 4 abstractly illustrates an image generation system with which theinteractive image projection system may operate;

FIG. 5 abstractly illustrates an embodiment of an input device of theembodiment of distributed system of FIG. 1;

FIG. 6 illustrates a specific embodiment of an input device;

FIG. 7 illustrates another specific embodiment of an input device;

FIG. 8 is a flowchart of a method for projecting and detectinginteraction with a projected image;

FIGS. 9 and 10 illustrate specific embodiments of interactive imageprojection systems; and

FIG. 11 illustrates a computing system architecture in which theprinciples described herein may be employed in at least someembodiments.

DETAILED DESCRIPTION

The principles described herein relate to the projection of an image toan interactive environment. The image may be two-dimensional or it maybe three-dimensional. The image may include one or more virtual objectsthat are spatially positioned within a virtual environment space. Theimage is projected to provide a visual representation of all or aportion of the virtual environment space, including a visualrepresentation of one or more of the virtual objects. The interactiveimage projection system may then detect user interaction with the imageand, thus, with the visual representation of the virtual environmentspace and, in response to any detected interaction, change the image,and perhaps cause a change (e.g., a temporary change, a permanentchange, etc.) to a state of a program or an application corresponding tothe image that has been projected (e.g., for which the image provides agraphical user interface (GUI), etc.).

Although not required, the input mechanism may be especially useful in adistributed system 100, such as a distributed electronic game system.FIG. 1 abstractly illustrates a distributed system 100. The distributedsystem 100 includes an interactive image projection system 101. Theinteractive image projection system 101 projects an image 111. Throughunique features of the distributed system 100 described hereinafter, auser may interact with the image 111 that has been projected.

In some embodiments, the image 111 may be projected onto a surface. Thesurface may be opaque or translucent (e.g., frosted glass, etc.).

The surface may be a substantially horizontal surface, in which case theimage 111 may be projected, at least in part, downward onto the surface.As an example, the substantially horizontal surface may be a table top,a counter top, a floor, a game board, or any other surface that isoriented substantially horizontally. A “substantially horizontal”surface may be any surface that is within 30 degrees of horizontal. A“more precisely horizontal” surface may be any surface that is within 5degrees of horizontal. Alternatively, the surface may be a substantiallyvertical surface. A “substantially vertical” surface may be any surfacethat is within 30 degrees of vertical. A “more precisely vertical”surface may be any surface that is within 5 degrees of vertical. Asanother alternative, the surface may be oriented orthogonally (i.e., ata non-parallel, non-perpendicular angle to the horizon).

In another embodiment, the image 111 may be projected onto a morecomplex surface. For instance, the surface onto which the image 111 isprojected may include one or more substantially horizontal surfaces, oneor more substantially vertical surfaces, and/or one/or more orthogonallyoriented surfaces. As an example, the complex surface might include, asa substantially horizontal surface, all or part of a surface of a floor,a table, a game board, or the like, and, as a substantially verticalsurface, all or part of a wall, a projection screen, a white board, orthe like. Other examples of complex surface may include texturedsurfaces, curved surfaces, and surfaces with nonplanar topographies.

The image 111, as projected by the interactive image projection system101, represents an interactive environment space in which one or moreusers may interact with the image 111 or features thereof (e.g., virtualobjects, etc.). One or more users may interact with the image 111manually, with physical objects, and/or with one or more input devices.The image 111 might be projected to a collaborative area, a work area,or any other type of interactive area. However, in the remainder of thisdescription, the distributed system 100 is often described as being agame or as being used in conjunction with a game. In those cases, eachuser would be a player, and the interactive environment space to whichthe image 111 is projected would be an interactive play space. Theprinciples described herein may apply to any environment in which one ormore users interact with a projected image.

Since FIG. 1 is in abstract, the interactive image projection system 101and the image 111 are only depicted as abstract representations.Subsequent figures will illustrate more specific embodiments of theinteractive image projection system 101 and the image 111.

Optionally, the distributed system 100 may also include surroundingcontrol devices, which are also referred to herein as “input devices.”There are eight input devices 102A-H illustrated in FIG. 1, although theellipses 1021 represent that a distributed system 100 may include fewerthan eight input devices 102A-H or more than eight input devices 102A-H.The input devices 102A-H are represented abstractly as rectangles,although each will have a particular concrete form depending on itsfunction and design. Example forms of input devices 102A-H are describedin further detail below. In the context of a game, for example, theinput devices 102A-H may be player consoles. However, the inclusion ofone or more input devices 102A-H in a distributed system 100 isoptional.

As an alternative to providing input through the input devices 102A-H,each user may instead provide input through direct, physical interactionwith a three-dimensional space adjacent to a location to which the image111 is projected. Such direct interaction may be provided for example,with a hand and one or more fingers, by manipulating physical objects(e.g., game pieces, etc.) positioned in relation to the image 111, or,perhaps, by rolling dice or playing cards in association with the image111. The interactive image projection system 101 is capable ofresponding to multiple simultaneous instances of users interacting witha location to which the image 111 is projected. Thus, input into thedistributed system 100 may be achieved using one or more input devices102A-H and/or by direct interaction with the interactive environmentimage 111. Thus, a user may affect the state of the image 111 and/or ofa program or an application associated with the image 111 (e.g., forwhich the image provides a GUI).

In one embodiment, one, some, or even all of the input devices 102A-Hare wireless. In the case of a wireless input device 102A-H, thewireless input device 102A-H may communicate wirelessly with theinteractive image projection system 101. One or even some of the inputdevices 102A-H may be located remotely from the image 111. Such remotelylocated game input device(s) 102A-H may communicate with the interactiveimage projection system 101 over a Wide Area Network (WAN), such as theInternet. That would enable a user to interact with the image 111remotely, even if that user is not located in proximity to the image111. Thus, for example, a father or mother stationed overseas might playa child's favorite board game with their child before going to bed. Orperhaps former strangers and new friends from different cultures aroundthe globe might engage in a game, potentially fostering cross-culturalties while having fun. That said, perhaps all of the game input devices102A-H may be local (e.g., in the same room, etc.) to the interactiveimage projection system 101. In yet another embodiment, there are noinput devices 102A-H. Regardless of whether there are input devices102A-H or not, the user might directly interact with the image 111.

FIG. 2 abstractly illustrates an interactive image projection system 200that represents an embodiment of the interactive image projection system101 of FIG. 1. The interactive image projection system 200 isillustrated as including an output channel 210 that projects an image(e.g., the image 111 of FIG. 1). The output channel 210 includes severalfunctions including image preparation and projection. Image preparationis performed by an image preparation mechanism 211, and projection ofthe image 111 is performed by projector(s) 212A, 212B, etc., with oneprojector 212A being depicted and the ellipses 212B representing one ormore optional additional projectors in the output channel 210 of theinteractive image projection system 200.

The image preparation mechanism 211 receives an input image 201 andsupplies an output image 202 in response to receiving the input image201. The input image 201 may be provided by any image generator. As anexample, the input image 201 might be provided by a video game console,a rendering program (whether two dimensional or three-dimensional), orany other module, component or software, that is capable of generatingan image.

The input image 201 represents one or more virtual objects that arepositioned in a virtual environment space. As an example, the virtualenvironment space may represent a battleground with specific terrain.The battleground is represented in a computer, and need not representany actual battleground. Other examples of virtual environment spacemight include a three-dimensional representation of the surface of theMoon, a representation of a helium atom, a representation of a crater ofa fictional planet, a representation of a fictional spacecraft, arepresentation of outer space, a representation of a fictionalsubterranean cave network, and so forth. Whether representing somethingreal or imagined, the virtual environment space may be embodied by acomputer program or an application.

Virtual objects may be placed in the virtual environment space by acomputer program or an application, and may represent any object, realor imagined. For instance, a virtual object might represent a soldier, atank, a building, a fictional anti-gravity machine, or any otherpossible object, real or imagined.

FIG. 3 illustrates an example of a virtual environment space 300. Inthis example, the virtual environment space 300 includes virtual objects301, 302, 303, and 304. In this case, the virtual environment space 300is a three-dimensional space, such that the virtual objects 301, 302,303, and 304 are represented as three-dimensional objects havingspecific shapes, positions, and/or orientations within the virtualenvironment space 300. This virtual environment space 300 may be used inorder to formulate a representation of a certain portion and/orperspective of the virtual environment space 300. The output image 202,as projected, includes a visual representation of at least part of thevirtual environment space 300, the visual representation includes avisual representation of at least one of the virtual objects 301, 302,303, and 304. For instance, if the virtual environment space 300comprises the inside of a virtual crater, the output image 202 mayprovide a visual representation of at least a portion of that crater,with virtual objects 301, 302, 303, and 304 that might include severalcrater monsters, soldiers that are members of the same team, weaponsthat are strewn about and ready to be picked up, and so forth. If thevirtual environment space 300 were a city, the visual representationmight be a portion of the city and include virtual objects 301, 302,303, and 304 that comprise things like vehicles, buildings, people, andso forth.

With returned reference to FIG. 2, the image preparation mechanism 211may perform any processing on the input image 201 to generate the outputimage 202 that is ultimately projected by the one or more projectors212A, 212B. As an example, the image preparation mechanism 211 maysimply pass the input image 201 through, such that the output image 202is identical to the input image 201. The image preparation mechanism 211might also change the format of the input image 201, change theresolution of the input image 201, compress the input image 201, decryptthe input image 201, select only a portion of the input image 201, orthe like. If multiple projectors 212A, 212B are used, the imagepreparation mechanism 211 may select which portion of the input image201 (i.e., a “subimage”) is to be projected by each projector 212A,212B, such that when the output images 202 are projected by eachprojector 212A, 212B, the collective whole of all of the output images202 may appear as a single image at the location to which the outputimages 202 are projected, a process referred to herein as “stitching.”

The image preparation might also take into consideration appropriateadjustments given the surface on which the output image 202 is to beprojected, or any intervening optics. For instance, if the output image202 is to be projected onto a complex surface, the image preparationmechanism 211 may adjust the input image 201 such that the output image202 will appear properly on the complex surface. The user mightconfigure the image preparation mechanism 211 with information regardingthe complex surface. Alternatively, or in addition, the interactiveimage projection system 200 may be capable of entering a discovery phaseupon physical positioning that identifies the characteristics of anysurface onto which an output image 202 is to be projected, in relationto the projector(s) 212A, 212B. As an example, if the surface includes acombination of horizontal, orthogonal, and/or vertical surfaces, theimage preparation mechanism 211 may take into consideration thedistances to the surfaces and the angles at which the surfaces areoriented to make sure that the output image 202 appears proportional andas intended on each surface. Thus, the image preparation mechanism 211may make appropriate geometrical adjustments to the input image 201 sothat the output image 202 appears properly on each surface. Otherexamples of complex surfaces include spherical surfaces, full andpartial cylindrical surfaces, surfaces that include convex portions,surfaces that include concave portions, other curved surfaces, includingsurfaces with repeated curvatures or other complex curvatures, andsurfaces that represent a nonplanar topography (as in a complex terrainwith various peaks and valleys). In cases in which the output image 202is to pass through optics such as lens and mirrors, the imagepreparation mechanism 211 may consider the presence of such optics andmodify the input image 201 accordingly.

In addition to image preparation and projection, the interactive imageprojection system 200 may also output various signals. For instance, theinteractive image projection system 200 may output audio, such as audiothat corresponds to the input image 201. The interactive imageprojection system 200 may output wired or wireless signals to the inputdevices 102A-H, perhaps causing some private state to be altered at theinput devices 102A-H. In addition, if there is a central display thatdisplays an image (e.g., the interactive central display described inthe co-pending commonly assigned application Ser. No. 12/411,289, etc.)(hereinafter referred to simply as the “central display”), theinteractive image projection system 200 may dispatch information in awired or wireless fashion to the central display.

User input may be provided through interaction with an input device(such as one of the input devices 102A-H of FIG. 1) and/or throughdirect interaction of a real object (such as a human finger, a gamepiece, a game board, a central display or the like) with athree-dimensional space adjacent to a location where the output image202 is projected. If there is to be direct interaction to provide input,the interactive image projection system 200 may also include an inputchannel 220.

The input channel 220 may include a scanning mechanism 221 capable ofscanning a three-dimensional space adjacent to a location where theoutput image 202 is projected to determine whether or not a user isinteracting with a virtual object displayed by or as part of the outputimage 202 or with another object (e.g., a physical object, etc.) thatthat may be used in conjunction with the output image 202. Morespecifically, the scanning mechanism 221 may detect movement of amanipulating element (e.g., a physical object, such as finger, a thumb,a hand, an object held by a user, etc.) into and within thethree-dimensional space adjacent to a location where the output image202 is projected.

As an example, suppose that the output image 202 of FIG. 2 includes justtwo-dimensional information. In that case, the projector(s) 212A, 212Bproject(s) the frame of the output image 202 or each output image 202.Then, after that frame or output image 202 is projected, during a shortperiod before the next frame or output image 202 is projected, thescanning mechanism 221 may scan the area where the last frame or outputimage 202 was projected. This projection and scanning process is thenrepeated for the next frame output image 202, and for the subsequentframe or output image 202, and so on. Even though projection andscanning may not happen at the same time (with scanning happeningbetween projection of sequential frames or an output image 202 orbetween projection of sequential output images 202), they happen at sucha high frequency that the output image(s) 202 may seem to havecontinuous motion. Furthermore, even though the frame or output image202 may not always be present, the period of time that the frame or theoutput image 202 is not present may be so short, and occur at afrequency that it may provide a human observer with the illusion thatthe frame or output image 202 is always present. Thus, real objects mayhave the appearance of occupying the same space as the output image(s)202. Alternatively, the scanning mechanism 221 may operate while anoutput image 202 or a frame thereof is projected and displayed.

As another example, the output image 202 of FIG. 2 may representthree-dimensional information. In that case, for each frame of theoutput image 202 or from each sequence of output images 202, theprojector(s) 212A, 212B may project a left eye image intended for theleft eye, and a right eye image intended for the right eye. Whenappropriate aids are present that allow the left eye of a human observerto receive the left eye image (but not the right eye image), and thatallow the right eye of that same human observer to receive the right eyeimage (but not the left eye image), the output image 202 can be observedby the human mind as being truly three dimensional. Three-dimensionalglasses are an appropriate aid for enabling this kind of eye-specificlight channeling, but the principles of the present invention are notlimited to the type of aid used to allow a human observer toconceptualize three-dimensional image information.

In one example, projection of the left eye image and projection of theright eye image are interlaced, with each being displayed at a frequencyat which continuous motion is perceived by a human observer. Typically,an average human observer cannot distinguish discrete changes, butinstead perceives continuous motion, between frames that are output at afrequency of at least 44 frames per second. Thus, a system that operatesat 120 Hz, and which interlaces a left eye image and a right eye image,each at 60 Hz, will suffice to formulate the appearance of continuousthree-dimensional motion. At periodic times, the scanning mechanism 221may scan for any objects (e.g., manipulating elements, etc.) that moveinto and/or within a three-dimensional space adjacent to a locationwhere the output image 202 is projected in a manner that may compriseinteraction of such an object with the output image 202. In aninteractive image projection system 200 that operates at a frequency of120 Hz, for example, the scanning may also occur at a frequency of 120Hz, at a frequency of 60 Hz, or at some other interval. That said, theprinciples described herein are not limited to any particular frame ratefor projection and sampling rate for scanning.

The input channel 220 of the interactive image projection system 200 mayalso include an input preparation function provided by, for example, aninput preparation mechanism 222. This input preparation mechanism 222may take the input provided through the scanning process and provide itin another form recognizable by a system that generates an input image201 (such as perhaps by a conventional video game system). For instance,the input preparation mechanism 222 may receive information from thescanning mechanism 221 that allows the input preparation mechanism 222to recognize gestures and interaction with virtual objects that aredisplayed and that may be visualized by one or more users. The inputpreparation mechanism 222 might recognize the gesture, and correlatethat gesture to particular input. The input preparation mechanism 222may consider the surface configuration, as well as any optics (such asmirrors or lenses) that may intervene between the location(s) to whichthe output image(s) 202 is (are) projected and the scanning mechanism221.

As an example, suppose that the output image 202 is of a game board,with virtual game pieces placed on the game board. The user might reachinto the output image 202 to the location of a virtual game piece (e.g.,by simulated touching since the virtual game piece cannot be touched orotherwise physically contacted), and “move” that virtual game piece fromone location of the game board to another, thereby advancing the stateof the game, perhaps permanently. In that case, the movement may occurover the course of dozens or even hundreds of frames or output images202, which, from the user's perspective, occurs in a moment. The inputpreparation mechanism 222 recognizes that a physical object (e.g., amanipulation element, etc., such as a human finger) has reached into athree-dimensional space adjacent to a location to which the output image202 is projected, and extended to or adjacent to the location where thevirtual game piece appears. If the image were a three-dimensional image,the input preparation mechanism 222 could monitor the position of thephysical object in three-dimensional space relative to athree-dimensional position of the virtual game piece. The virtual gamepiece may comprise a projected portion of the output image 202 and,thus, the user would not feel the virtual game piece, but the inputpreparation mechanism 222 may recognize that the user has indicated anintent to perform some action on or with the virtual game piece.

In subsequent frames or output images 202, the input preparationmechanism 222 may recognize slight incremental movement of the physicalobject, which may represent an intent to interact with the output image202 or a feature of the output image 202 (e.g., to move a virtual gamepiece in the same direction and magnitude as the finger moved, tootherwise manipulate the output image 202, etc.) and, optionally, anintent to interact with the output image or a feature thereof in aparticular manner. The input preparation mechanism 222 may issuecommands to cause the image preparation mechanism 211 to modify theoutput image 202 (now an input image 201) in an appropriate manner(e.g., to cause the virtual game piece to move in the virtualenvironment space, etc.). The changes can be almost immediately observedin the next frame or in the next output image 202. This occurs for eachframe or output image 202 until the user indicates an intent to nolonger move the game piece (e.g., by tapping a surface on which theoutput image 202 is projected at the location at which the user wishesto deposit the virtual game piece, etc.).

The appearance to the player would be as though the player had literallycontacted the virtual game piece and caused the virtual game piece tomove, even though the virtual game piece is but a projection.Accordingly the interactive image projection system 200 may enable theprojection and movement of virtual objects or otherwise enable theprojection and manipulation of a virtual environment space. Otheractions might include resizing, re-orienting, changing the form, orchanging the appearance of one or more virtual objects with which a userinteracts.

As a further example, the user may interact with physical objectsassociated with an image that has been projected. The input channel 220may recognize the position, orientation, and/or configuration of thephysical object and interpret user movements and/or gestures (e.g.,movement or manipulation of a physical object, etc.) or interaction withvirtual features associated with the physical object. For instance, inthe MONOPOLY board game, a physical game board may be placed within aprojected image that might include virtual objects, such as, virtual“Chance” and “Community Chest” cards, virtual houses and hotels, andperhaps a combination of real and virtual game pieces (according toplayer preference configured at the beginning of a game). A player mighttap on a property owned by that player, which the input channel mayinterpret as an intent to build a house on the property. The inputchannel 220 might then coordinate with any external image generationsystem and the output channel 210 to cause an additional virtual houseto appear on the property (with perhaps some animation). In addition,the input channel 220 may coordinate to debit the account of that playerby the cost of a house. In addition, information may be transmitted to apersonal input device 102A-H operated by the user to update an accountbalance displayed by the personal input device 102A-H.

As another example of the MONOPOLY board game, the player might rollactual dice at the beginning of the player's turn. The input channel 220may recognize the numbers on the dice after they have been rolled andcause the projected image to highlight the position that the player'sgame piece should move to. If the player has a virtual game piece, thenthe system might automatically move (with perhaps some animation) thevirtual game piece, or perhaps have the user move with the player'sinteraction with the virtual game piece (perhaps configured by the userto suit his/her preference). In response, the interactive imageprojection system 200 might transmit a prompt to the user's input device102A-H, requesting whether the user desires to purchase the property, ornotifying the user of rent owed. In one embodiment, the output channel210 not only projects images, but also responds to an external gamesystem to provide appropriate output to appropriate devices. Forinstance, the output channel 210 might recognize that the external gamesystem is providing the current player with an inquiry as to whether ornot the current player wants to purchase the property. The outputchannel 210, in addition to projecting the appropriate image, may alsotransmit an appropriate prompt to the player's input device 102A-H.

In yet a further example, a central display may display an image and bepositioned within an image that has been projected by the interactiveimage projection system 101. Thus, a projected image may be superimposedwith an image displayed by the central display.

In some embodiments, the principles described herein may take aconventional system and allow for a unique interaction with a projectedimage. The interactive image projection system 200 may interface with aconventional image generation system (e.g., a graphic processor, etc.)to enable interaction with an image that has been projected. Theinteractive image projection system 200 may receive an image generatedby the conventional image generation system, with the image preparationmechanism 211 conducting any processing of any interaction by a userwith the projected image. The conventional image generation system maygenerate the image in the same manner as if the image were just to bedisplayed by a conventional display or projector. Once a user hasinteracted with a projected image and such interaction has been detectedand processed by the image preparation mechanism 211, the conventionalimage generation system receives commands from the image preparationmechanism 211 as it is accustomed to receive commands from conventionalinput devices to effect a change in the game state of a program or anapplication for which a GUI has been displayed (i.e., projected) andadvance use of the program or the application. The conventional imagegeneration system may operate in the same manner it would normallyfunction in response to conventional inputs, no matter how complex thesystems used to generate the commands. Whether the input was generatedby a conventional hand-held controller, or through the complexity of theinput channel 220, the conventional image generation system will operatein its intended manner.

In addition to being capable of preparing input information forconventional image generation systems, the input channel 220 may provideinformation for other surrounding devices, such as, any of one or moreconventional input devices, and perhaps a central display, associatedwith the conventional image generation system, thereby altering state ofany of these devices, and allowing for these devices to participate ininteracting with the program or the application whose outputs are beinginteractively projected.

FIG. 4 abstractly illustrates an image generation system 400, which maybe used to generate the input image 201 of FIG. 2. In one embodiment,the image generation system 400 may be a conventional video game thatoutputs an image that might, for example, change as a player progressesthrough the video game. However, one, some, and perhaps even all of thefunctions described as being included within the image generation system400 may be performed instead within the interactive image projectionsystem 101.

The image generation system 400 includes logic 411, an image generationmechanism 412, and an input interface 413. The logic 411 and/or theimage generation mechanism 412 control a virtual environment space. Theimage generation mechanism 412 generates an image that is appropriategiven a current state 414 of the logic 411 and, thus, of the virtualenvironment space. The input interface 413 receives commands that mayalter the state 414 of virtual environment space and, thus, of the logic411, thereby potentially affecting the image generated by the imagegeneration mechanism 412. The state 414 may even be altered from onestage to the next as one or more users interact with a program or anapplication through the input interface 413. In such systems, images canbe generated at such a rate that continuous motion is perceived. Theremay be a bi-directional channel of communication 1108 (FIG. 11) betweenthe image generation system 400 and the interactive image projectionsystem 200. The bi-directional channel may be wired or wireless, orperhaps wired in one direction and wireless in another. Input commandsare typically less data-intensive as compared to images, and thus thechannel of communication 1108 from the interactive image projectionsystem 200 to the image generation system 400 may be wireless. Thechannel of communication 1108 from the image generation system 400 tothe interactive image projection system 200 may also be wirelessprovided that the bandwidth of the channel in that direction issufficient.

The interactive image projection system 101 and/or any associated inputdevices 102A-H may have built-in microphones to allow sound data (e.g.,the player's voice, etc.) to be input into the image generation system400 to affect the state 414. There may also be voice recognitioncapability incorporated into the interactive image projection system 101and/or any associated input devices 102A-H to permit such sound data tobe converted to more usable form. Speakers, headset ports, and earpiecesmay be incorporated into the interactive image projection system 101and/or into any input devices 102A-H associated with the interactiveimage projection system 101.

FIG. 5 abstractly illustrates an embodiment of a player console 500. Aspreviously mentioned, the input devices 102A-H of FIG. 1 may be playerconsoles in the context in which the distributed system 100 is a gameenvironment. FIG. 5 is an abstract illustration of a player console 500showing functional components of the player console 500. Each player, orperhaps each team of players, may have an associated player console,each associated with the corresponding player or team. The playerconsole 500 includes a private display area 501 and game logic 502capable of rendering at least a portion a private portion of game state503 associated with the player (or team). The player or team may use aninput mechanism 504 to enter control input into the player console 500.A transmission mechanism illustrated in the form of a transceiver 505transmits that control input to the interactive image projection system200 of FIG. 2 and/or to the image generation system 400 of FIG. 4, wherethe control input is used to alter the state 414 of the logic 411 usedto generate the image.

FIG. 6 illustrates a specific embodiment of a player console 600. Here,the private display area 601 displays the player's private information(in this case, several playing cards). The player console 600 alsoincludes a barrier 602 to prevent other players from seeing the privategame state displayed on the private display area 601. The privatedisplay area 601 may be touch-sensitive, allowing the player to interactwith physical gestures on the private display area 601, thereby causingcontrol information to update the rendering on the private display area601, and the game states on the player console 600, as well as on thecentral display 101. The private display area 601 may also display videoimages 603A, 603B, and 603C of other players.

In one embodiment, at least one of the player consoles is different fromthe remaining player consoles 600. FIG. 7 illustrates such a playerconsole, which might be a game master console 700, with which a gamemaster may interface with the private viewing area to perhaps controlgame state. For instance, the game master may use physical gestures on atouch-sensitive display 701 of the game master console 700 to affectwhat is displayed within the image 111. For instance, the game mastermight control what portions of the map are viewable in the image 111.The game master might use the game mater control 700 to control theeffect of another player's actions on the operation of the game logic.The game master might also use the game master console 700 to create ascenario and to set up a game.

FIG. 8 is a flowchart of a method 800 for projecting an image and forenabling interaction with the image. At reference 801, data representingone or more virtual objects that are spatially positioned in a virtualenvironment space is received. An example of such data is an image inwhich such virtual objects are represented. The image is then projectedat reference 802 in response to the received data. The image may providea visual representation of at least part of the virtual environmentspace. At reference 803, any user interaction with the visualizedrepresent provided by the image may be detected. In response to thatuser interaction, the projected image is then altered at reference 804.

FIG. 9 illustrates an embodiment of an interactive image projectionsystem 900 in which multiple modules 902A through 902E are mounted to astand 901. Each module 902A through 902E includes a projector and acorresponding camera (not shown) which would be in the lower surface ofeach module 902A through 902E. The projector projects the imagesdownward towards a surface on which the stand 901 is situated. Theseprojectors would each project a corresponding subimage that are eachprocessed such that the projected image is stitched together to appearas a single image on or over the surface. The camera scans for userinteraction in the area of the image that has been projected.

FIG. 10 illustrates another embodiment of an interactive imageprojection system 1000 that includes a single projector. The interactiveimage projection system 1000 includes a housing that includes a rigidbase 1001 situated on a substantially horizontal surface. A projector1011 is capable of projecting an image upward through a lens to a curvedmirror 1012, from which the image is reflected and projected throughwindows 1013, and the projected downward onto the substantiallyhorizontal surface on which the base 1001 is placed. The images aregenerated to account for the intervening lens(es), mirror(s) 1012, andwindow(s) 1013 used to project the image. Four cameras (of which three1021A through 1021C are visible in FIG. 10) are positioned around theupper circumference of the interactive image projection system 1000.Such cameras 1021A through 1021C are capable of scanning athree-dimensional space adjacent to a location to which the image isprojected to detect any interaction with the image.

The various operations and structures described herein may, but neednot, be implemented by way of a physical computing system. Accordingly,to conclude this description, an embodiment of a computing system willbe described with respect to FIG. 11. The computing system 1100 may beincorporated within the interactive image projection system 101, withinone or more of the input devices 102A-H, and/or within the imagegeneration system 400.

Computing systems are now increasingly taking a wide variety of forms.Computing systems may, for example, be handheld devices, appliances,laptop computers, desktop computers, mainframes, distributed computingsystems, or even devices that have not conventionally been considered tobe computing systems. In this description and in the claims, the term“computing system” is defined broadly as including any device or system(or combination thereof) that includes at least one processor and memorycapable of having thereon computer-executable instructions that may beexecuted by the processor(s). The memory may take any physical form andmay depend on the nature and form of the computing system. A computingsystem 1100 may communicate with other devices, including, but notlimited to other computing systems, over a network environment 1110,which may include multiple computing systems. In some embodiments,components of a single computing system 1100 may be distributed over anetwork environment 1110.

In its most basic configuration, a computing system 1100 may include atleast one processor 1102 and memory 1104. The memory 1104 may comprise aphysical system memory, which may be volatile, non-volatile, or somecombination of the two. The term “memory” may also be used herein torefer to non-volatile mass storage such as physical storage media. Ifcomponents of the computing system 1100 are distributed over a networkenvironment 1110, the processor 1102, memory 1104, and/or storagecapability may be distributed as well. As used herein, the term “module”or “component” can refer to software objects or routines that execute onthe computing system 1100. The different components, modules, engines,and services described herein may be implemented as objects or processesthat execute on the computing system (e.g., as separate threads, etc.).

In the description above, embodiments are described with reference toacts that are performed by one or more computing systems. If such actsare implemented in software, one or more processors of the associatedcomputing system 1100 that performs the act direct the operation of thecomputing system 1100 in response to having executed computer-executableinstructions. An example of such an operation involves the manipulationof data. The computer-executable instructions (and the manipulated data)may be stored in the memory 1104 of the computing system 1100.

Embodiments within the scope of the present invention also includecomputer-readable media for carrying or having computer-executableinstructions or data structures stored thereon. Such computer-readablemedia can be any available media that can be accessed by a generalpurpose or special purpose computer. By way of example, and notlimitation, such computer-readable media can comprise physical storageand/or memory media such as RAM, ROM, EEPROM, CD-ROM or other opticaldisk storage, magnetic disk storage or other magnetic storage devices,or any other physical medium which can be used to carry or store desiredprogram code means in the form of computer-executable instructions ordata structures and which can be accessed by a general purpose orspecial purpose computer. Combinations of the above should also beincluded within the scope of computer-readable media.

Computer-executable instructions comprise, for example, instructions anddata which cause a general purpose computer, special purpose computer,or special purpose processing device to perform a certain function orgroup of functions. Although the subject matter has been described inlanguage specific to structural features and/or methodological acts, itis to be understood that the subject matter defined in the appendedclaims is not necessarily limited to the specific features or actsdescribed herein. Rather, the specific features and acts describedherein are disclosed as example forms of implementing the claims.

The components of the computing system 1100 may, for example, be used toprovide functionality to game logic, store or remember game state,configure and communicate between devices, and operate the logic of gameincorporation. Each of the player consoles may also have a computingsystem such as computing system 1100 guiding their processing needs.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed:
 1. A method for projecting an interactive environmentimage, comprising: displaying an environment image including at leastone virtual object; and detecting user interaction with athree-dimensional space corresponding to the environment image,including: scanning the three-dimensional space; and detectinginteraction between a manipulating element and a location in thethree-dimensional space corresponding to a location of the at least onevirtual object.
 2. The method of claim 1, wherein detecting the userinteraction comprises detecting a physical movement or manipulation of aphysical object within the three-dimensional space.
 3. The method ofclaim 1, wherein detecting the user interaction further includes:detecting extension of the manipulating element from a location outsideof the three-dimensional space into the three-dimensional space.
 4. Themethod of claim 3, wherein detecting the user interaction comprisesdistinguishing between a plurality of elements simultaneously extendingfrom locations outside of the three-dimensional space into thethree-dimensional space.
 5. The method of claim 1, further comprising:changing the environment image in response to interaction between themanipulating element and the three-dimensional space.
 6. The method ofclaim 5, wherein changing the visualized representation comprisesaffecting the at least one virtual object.
 7. The method of claim 6,wherein affecting the at least one virtual object comprises moving,manipulating, or changing the at least one virtual object.
 8. The methodof claim 5, wherein changing the visualized representation comprisesadding a new virtual object to the visualized representation.
 9. Themethod of claim 1, wherein displaying the environment image includesdisplaying at least a portion of the environment image on a surface. 10.The method of claim 9, wherein displaying the environment image includesdisplaying at least the portion of the environment image on asubstantially flat surface.
 11. The method of claim 9, whereindisplaying the environment image comprises displaying a prepared imagetailored to a topography of the surface to ensure that features of theenvironment image are proportionately displayed on the surface.
 12. Themethod of claim 9, wherein detecting user interaction includes detectinguser interaction with a topography of the surface.
 13. The method ofclaim 1, wherein the environment image includes three-dimensionalinformation viewable by a user as a three-dimensional image.
 14. Themethod of claim 1, wherein displaying the environment image comprisesdisplaying a plurality of discrete sub-images.
 15. A computing system,comprising: a display that displays an environment image including atleast one virtual object; and a scanning mechanism that detects a userinteraction with the environment image by: scanning a three-dimensionalspace corresponding to the virtual environment space; and detectinginteraction between a manipulating element and a location within thethree-dimensional space that corresponds to a location of the at leastone virtual object.
 16. The computing system of claim 15, wherein thescanning mechanism is detects interaction by: detecting extension of themanipulating element from a location outside of the three-dimensionalspace into the three-dimensional space.
 17. The computing system ofclaim 15, further comprising: at least one processing element programmedto change the environment image in response to the user interaction withthe three-dimensional space.
 18. The computing system of claim 17,wherein the scanning mechanism comprises the at least one processingelement and at least one camera.
 19. A portable display system,comprising: at least one display that displays an environment imagerepresenting at least one virtual object positioned in a virtualenvironment space; and a scanning mechanism that scans athree-dimensional space corresponding to the virtual environment spaceto determine one or more locations of interactivity by: scanning thethree-dimensional space that corresponds to the virtual environmentspace of the environment image; and detecting interaction between amanipulating element and a location within the three-dimensional spacethat corresponds to a location of a virtual object of the plurality ofvirtual objects within the virtual environment space.
 20. The portabledisplay system of claim 19, wherein the scanning mechanism further:detects extension of the manipulating element into the three-dimensionalspace from a location outside of the three-dimensional space.